Large margin learning of hierarchical semantic similarity for image classification

In the present paper, a novel image classification method that uses the hierarchical structure of categories to produce more semantic prediction is presented. This implies that our algorithm may not yield a correct prediction, but the result is likely to be semantically close to the right category. Therefore, the proposed method is able to provide a more informative classification result. The main idea of our method is twofold. First, it uses semantic representation, instead of low-level image features, enabling the construction of high-level constraints that exploit the relationship among semantic concepts in the category hierarchy. Second, from such constraints, an optimization problem is formulated to learn a semantic similarity function in a large-margin framework. This similarity function is then used to classify test images. Experimental results demonstrate that our method provides effective classification results for various real-image datasets. Recognizing categories of objects and scenes is one of the most critical problems in computer vision. Although continuous progress has been made in this field, there still remains a large gap between machine performance and human intelligence. Unlike machines, humans can categorize at least tens of thousands of objects and scenes without any difficulty [1]. Furthermore, they can build a hierarchy of categories by simply observing images, and exploit it to produce semantically more meaningful judgement. For example, someone may mistakenly classify a dog as a cat but hardly misclassify a dog as a car. This example shows that one can produce a more informative classification result by considering the similarity between two semantic concepts. In the current work, we focus on this issue and attempt to make the image classification algorithm more semantic and human-like. To achieve this goal, image classification algorithm should be developed under a new performance evaluation criterion. This criterion can be formulated by utilizing the hierarchical loss, which reflects the hierarchy of various semantic concepts, and not the flat 0/1 loss. Similar to [2], the hierarchical loss can be defined based on WordNet [3], a lexical semantic network for modeling human psycholinguistic knowledge. Under the hierarchical loss-based criterion, misclassifying an image as a different but semantically close category incurs a smaller loss than misclassifying it as a semantically distant category. Therefore, image classification can be significantly more informative by learning the algorithm based on the hierarchical loss. As shown in Fig. 1, the use of the hierarchical loss can provide substantial benefit to the results of classification. …